# Multilingual mixed training

The Gemma-2-Llama-Swallow series of models are multilingual large models constructed through continuous pre-training based on Gemma-2, with a particular enhancement in Japanese ability.

Stockmark-2-100B is a 100-billion parameter large language model focused on Japanese capabilities, pre-trained on 1.5 trillion tokens of multilingual data and enhanced with Japanese synthetic data for improved instruction following.

This model is a multilingual model fine-tuned on Swiss German text data through continual pre-training based on xlm-roberta-base, supporting both Swiss German and Standard German processing.

The first model in the OpenHathi series, based on the Llama2 architecture, supports Hindi, English, and mixed languages, with 7 billion parameters.

ByT5-Korean is a customized Korean extension of Google's ByT5, specifically optimized for Korean syllable encoding processing.